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We enumerate three specific permutation classes defined by two forbid- 
den patterns of length four. The techniques involve inflations of geomet- 
ric grid classes. 



1. Introduction 

Classes of permutations are sets that are closed downwards under taking subpermutations. They 
are often presented as sets C that avoid a given set B of permutations (i.e. the members of C have 
no subpermutation in the set B). We express this by the notation C = Av(B). We may take B to be 
an antichain (a set of pairwise incomparable permutations), in which case we say that B is the basis 
ofC. 

Much of the inspiration for the early work on permutation classes was driven by the enumer- 
ation problem: given C — Av(B), how many permutations of each length does C contain? The 
answer to such a question could be a formula giving this number \C n | in terms of the length, n, a 
generating function \C n \x n or simply an asymptotic result about the behaviour of \C n \ as n —> oo. 

Recently, Albert, Atkinson, Bouvel, Ruskuc, and Vatter [2J have developed the theory of geomet- 
ric grid classes, and Albert, Ruskuc, and Vatter [6J have continued this exploration by investigating 
the theory of inflations of such classes. Our aim in this paper is to: 

• demonstrate the effectiveness of this approach, and 

• illustrate how one might implement these techniques in a "real world" setting, bypassing 
what would otherwise be thorny theoretical issues. 



* Vatter 's research was sponsored by the National Security Agency under Grant Number H98230-12-1-0207. The United 
States Government is authorized to reproduce and distribute reprints not-withstanding any copyright notation herein. 
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It should be noted that this presentation is historically backward; the results of this paper preceded 
and inspired the two more theoretical papers cited above. 

In this work, our examples are exclusively classes with two basis elements of length four, which 
we call 2x4 classes. It must be admitted that the attention paid to 2 x 4 classes is not entirely in 
proportion to their intrinsic importance. Nevertheless, these classes represent a significant dataset 
which seems to contain some difficult enumerative problems. Thus they pose a good challenge for 
new approaches to the enumeration of restricted permutations. 

There are 56 essentially different (i.e. inequivalent under symmetries) 2x4 classes. Some of 
these classes nevertheless share the same enumeration (a phenomenon called Wilf-equivalence), so 
the 2x4 classes have only 38 different enumerations |8l fl4l [151 H6l IT7). This paper brings the 
number of 2 x 4 Wilf classes which have been enumerated to 24 (see Wikipedia 11221 , which contains 
a list of such enumerations). 

A central part of our approach depends on analysing the simple permutations in a class. An 
interval in the permutation tt is a set of contiguous indices I = {a, a + 1, . . . , b} such that the set 
: i £ 1} is also contiguous. Every permutation tt of length n has trivial intervals of lengths 0, 1, 
and n, and other intervals are called proper. A permutation with no proper intervals is called simple. 
Another way to think about simple permutations arises repeatedly throughout our arguments. Any 
subset pi, ■ ■ ■ of entries of the permutation tt defines a minimal axes-parallel rectangle (or simply, 
box), whose left edge slices through the leftmost of these entries, top edge slices through the greatest 
of these entries, and so on. A simple permutation is one in which the box defined by any proper 
subset of two or more of its entries is separated by an entry outside the box, by which we mean that 
this entry lies either 

• vertically amongst these entries but to the left (or right) of all of them (vertical separation), or 

• horizontally amongst these entries but above (or below) all of them (horizontal separation). 

Simple permutations are precisely those that do not arise from a non-trivial inflation, in the 
following sense. Given a permutation a of length m and nonempty permutations a\, . . . , a m , the 
inflation of a by cx\, . . . , a m , denoted o[a\,..., a m ], is the permutation of length |ai | + • • • + \a m \ 
obtained by replacing each entry a(i) by an interval that is order isomorphic to on in such a way 
that the intervals are order isomorphic to a. For example, 

2413[1, 132, 321, 12] = 4 798 321 56. 

We give two particular types of inflations special terminology and notation. The inflation 12[ai, 
is called a (direct) sum and denoted by ot\ © ct2- A permutation is sum decomposable if it can be 
expressed as a nontrivial sum, and sum indecomposable otherwise. The inflation 21 [ai, «2] is called a 
skew sum, similarly denoted a\ a2, and accompanied by analogous terms skew decomposable and 
skew indecomposable. We extend the notion of direct and skew sum to classes, defining 

C®V = {ir®<7 : tt eC and a G V}, 

with an analogous definition for C QT>. 

The precise connection between simple permutations and inflations is furnished by the follow- 
ing result. 

Lemma 1.1 (Albert and Atkinson [1]). For every permutation tt there is a unique simple permutation a 
such that tt = a[ai,ct2, . . . ,a m j. Furthermore, except when a = 12 or a = 21, the intervals of tt that 
correspond to a\, a%, . . . , a m are uniquely determined. In the case that a = 12 (respectively a = 21), 
the intervals are unique so long as we require the first of the two intervals to be sum (respectively skew) 
indecomposable. 



Inflations of Geometric Grid Classes: Three Case Studies 



3 



i i i i i i i i i i i i i i i I i i i i i i i i 

Figure 1: The four orientations of parallel alternations. 



One of the first general enumeration results is the following from 1 1 1: 

Theorem 1.2. If the class C contains only finitely many simple permutations, then C has an algebraic 
generating function. 

This theorem has since been generalised in two different directions. Brignall, Huczynska, and 
Vatter [9] introduced the notion of "query-complete sets of properties" to show that if a class sat- 
isfies the hypotheses of Theorem 11.21 then such subsets as the even permutations or the involu- 
tions in C have algebraic generating functions. More relevant to our investigation, [6] significantly 
weakened the hypotheses of Theorem ll.21 showing that its conclusion holds even when C contains 
infinitely many simple permutations, so long as these simple permutations lie in a geometric grid 
class, a notion introduced in Section[3] Before this, we consider an example which gives the flavour 
of our approach without requiring much additional machinery. 

2. Example #1: Avoiding 4213 and 3142 

Before describing our first example we need to introduce a family of simple permutations and quote 
a result. A parallel alternation is a permutation whose plot can be divided into two parts, by a single 
horizontal or vertical line, so that the points on either side of this line are both either increasing or 
decreasing and for every pair of points from the same part there is a point from the other part which 
separates them, i.e., there is a point from the other part which lies either horizontally or vertically 
between them. It is easy to see that a parallel alternation of length at least four is simple if and only 
if its length is even, it does not begin with its smallest entry, and it does not end with its greatest 
entry Thus there are precisely four simple parallel alternations of each even length at least six, 
shown in Figure[TJ and no simple parallel alternations of odd length. 

Schmerl and Trotter [19, Corollary 5.10] proved a result (in the more general context of irreflex- 
ive binary relational structures) which in our context states that every simple permutation of length 
n which is not a parallel alternation contains simple subpermutations of every length 5 < m < n. 
Therefore, in order to establish that the permutation class C contains only parallel alternations, we 
just need to check that it does not contain any simple permutation of length 5, i.e., that 

C C Av(24153, 25314, 31524, 35142, 41352, 42513). 

Clearly this holds for the class Av(4213, 3142), because 4213 is contained in 25314 and 42513 while 
3142 is contained in 24153, 31524, 35142, and 41352. Moreover, it is easily seen (because of the basis 
element 3142) that Av(4213, 3142) can contain only parallel alternations oriented as on the left of 
Figure[TJ i.e., those of the form 

246---(2m)135---(2m-l). 

With the simple permutations characterised, we now describe the allowed inflations. It is easy to 
see that tt © a € Av(4213, 3142) for all n, a g Av(4213, 3142), or in other words, that the class is sum 
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closed. Thus, letting / denote the generating function for nonempty permutations in Av(4213, 3142) 
and /e denote the generating function for sum decomposable permutations, we see that /® = 
(/ — /e) // from which it follows that 

P 

^ ~ 1 + /' 

For skew sums, we have that 7r a e Av(4213, 3142) if and only if w € Av(4213, 3142) and a G 
Av(213). Letting 

_ 1 - 2a; - a/1 ^ 4x 
C ~ 2a; 

denote the generating function for the Catalan numbers (with constant term zero), which is well- 
known as the generating function of nonempty permutations in Av(213), we have /e = (/ — /e) c > 
so 

Now we must count the inflations of the parallel alternations 246 • • • (2m)135 • • • (2m — 1) for 
m > 2. This is relatively straightforward: 

• the interval inflating 2m — 1 must avoid 213, 

• all other intervals inflating odd entries must be increasing, and 

• even entries may be inflated by any element of Av(4213, 3142). 

This shows that the contribution of inflations of 246 • • • (2m) 135 • • • (2m — 1), for each m > 2, is 

/ \ m— i 
/ t. \ 

r 



showing that 



1-x 



f cf xcf 1 



m=2 v 7 J 

From this we obtain: 

Theorem 2.1. The generating function f for Av(4213, 3142) satisfies 

x 3 f 6 + (7x 3 - 7a; 2 + 2x)f 5 

+ (a; 4 + 14a; 3 - 21a; 2 + lOx - l)/ 4 

+ (4a; 4 + 8a; 3 - 19a; 2 + llx- 2)/ 3 

+ (6a; 4 - 5a; 3 - 2a; 2 + 2a;)/ 2 

+ (4a; 4 - 7a; 3 + 4a; 2 -a;)/ 

+ a; 4 - 2a; 3 + a; 2 =0. 

The first several terms of this sequence are 

1, 2, 6, 22, 89, 379, 1664, 7460, 33977, 156727, 730619, 3436710, 16291842, 77758962, 

sequence A165541 in the OEIS [20|. The discriminant of the polynomial defining the generating 
function has a smallest positive root p s» 0.1895, which is therefore the radius of convergence of the 
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Figure 2: Examples of wedge simple permutations. 



generating function and as Av(4213, 3142) is sum closed (and hence the sequence /„ is supermulti- 

plicative) we can conclude that f}J n —t 1/p « 5.2778. More detailed on the asymptotic behaviour of 
/„ could be determined by standard methods as found for instance in Flajolet and Sedgewick fTTl 
Section VII. 7]. 

This is not the only 2x4 class to which such elementary techniques apply. For example: 

• Av(4213, 1342) contains precisely two simple permutations of each length n > 4, both of 
which are wedge simple permutations oriented as the first two permutations shown in Fig- 
ure 12 This family of simple permutations is well enough behaved that we could enumerate 
the class, but this has already been done by Kremer and Shiu [16J and can now be performed 
automatically using the Maple package FlNLABEL described in Vatter El . 

• Av(4213, 3124) contains precisely two simple permutations of each length n > 4, oriented as 
the rightmost two permutations shown in Figure 12 This class was enumerated by Bona [8 J. 



3. Grid Classes and Regular Languages 

Given a permutation tt of length n and sets X,Y C [a], we write tt(X x Y) for the permutation that 
is order isomorphic to the subsequence of tt with indices from X and values in Y. For example, 
286435179([4, 9] x [5,9]) consists of the subsequence of entries in indices 4 through 9 which have 
values between 5 and 9; in this case the subsequence is 579, so 286435179([4, 9] x [5, 9]) = 123. 

Suppose that M is a t x u matrbflwith entries from {0, ±1}. A gridded permutation is a permuta- 
tion tt equipped with row and column divisions denoted respectively by 1 = c\ < ■ ■ ■ < ct+i = n+1 
and 1 = n < • • • < r u +\ = n + 1 (where n is the length of tt). This gridded permutation (or simply, 
gridding of tt) is compatible with the matrix M (in which case we sometimes call it an M -gridding 
of tt) if 7r([cfe, Cfc+i) x [re., r^+i)) is increasing whenever Mk.i = 1, decreasing whenever Mk,t. = — 1, 
and empty whenever Mk,t = 0. The (monotone) grid class of M, written Grid(M), consists of all 
permutations which possess a gridding compatible with M. Figure |3] shows an example. 

As illustrated by Murphy and Vatter |18|, monotone grid classes can display chaotic and un- 
structured behaviour. However, it has recently been shown that these classes contain subclasses 
with especially amenable structure. To define these subclasses, consider the point set in M. 2 (called 
the standard figure of the 0/±l matrix M) consisting of cells Cm whose contents are: 

• the line segment from [k — 1, 1 — 1) to (fc, I) if M^.i = 1 or 

• the line segment from (k — 1, £) to (k, £ — 1) if Mk,e = — 1 or 

• empty if Mu = 0. 

1 Note that in order for the cells of the matrix M to be compatible with plots of permutations, we use Cartesian coordinates 
for our matrices, indexing them first by column, from left to right starting with 1, and then by row, from bottom to top. 
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Figure 3: The permutation 286435179 lies in the grid class of the matrix 



M = (" -I)' 

as the gridding on the left demonstrates. The figure in the center shows that the permutation 
17645328 lies in Geom(M). Finally, the figure on the right shows that 2413 does not lie in the 
geometric grid class of M (although it does lie in Grid(M)): traveling clockwise from 2, we see 
that 4 must lie closer to the centre than 2, 3 must lie closer to the centre than 4, but then 1 must 
lie closer to the centre than 3 but further from the centre than 2. 

The geometric grid class of M, denoted by Geom(M), is then the set of all permutations that can be 
drawn on this figure in the following manner. Choose n points in the figure, no two on a common 
horizontal or vertical line. Then label the points from 1 to n from bottom to top and record these 
labels reading left to right. The centre pane of Figure |3] shows a permutation from a geometric grid 
class, while the right pane demonstrates that 2413 is not in this geometric grid class. 

It sometimes happens that Grid(M) = Geom(M); to characterise this phenomenon, we need to 
introduce a graph. The row-column graph of a t x u matrix M is the bipartite graph on the vertices 
x\, . . . , x t , yi, . . . , y u where Xk is adjacent to yi if and only if Mk,i ^ 0. Albert, Atkinson, Bouvel, 
Ruskuc, and Vatter [2, Theorem 3.2] showed that Grid(M) = Geom(M) if and only if the row- 
column graph of M is a forest (in this case we say that M is a forest). As it happens, all gridding 
matrices encountered in this paper are forests. 

Geometric grid classes are especially tractable because their elements can be encoded by words 
over a finite alphabet, and for the rest of this section we describe this encoding and its properties. 
We say that a 0/±l matrix M of size t x u is a partial multiplication matrix if there exist column and 
row signs 

h,---,fu9i,---,9u e {1,-1} 

such that every entry Mk,i is equal to either fkge or 0. It is not hard to prove that every geometric 
grid class is equal to Geom(M) for a partial multiplication matrix M, and this is especially trivial 
for forests. 

The column and row signs essentially specify an order in which the monotone entries in a cell 
of a gridded permutation should be read. Cells corresponding to Mm — fkge are read from left to 
right (respectively right to left) if = 1 (respectively /& = —1) and bottom to top (respectively 
top to bottom) if gi = 1 (respectively gt = —1). These directions are sometimes marked on our 
diagrams. The base point of a cell is the corner from which its reading begins. 

To describe the encoding of Geom(M) we introduce a cell alphabet S associated to M which 
consists of a unique letter ati for each nonempty cell Cki of the standard figure of M. Then, to every 
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word w = w\ ■ ■ ■ w n € S* we associate a permutation tp{w). First we choose arbitrary distances 



For each 1 < i < n, we choose a point pi corresponding to Wi in the following manner. Let Wi — au', 
the point pi is chosen from the line segment in cell Ck,i, at infinity-norm distance di from the base 
point of this cell. Finally, ip(w) denotes the permutation defined by the set {pi, . . . ,p n } of points. 
It can be seen that tp does not depend on the particular choice of di, . . . , d n , and thus tp : S* — > 
Geom(M) is a well-defined mapping. 

The mapping tp is many-to-one, and so for enumerative applications we must restrict its domain 
to a set L C E* on which tp is injective. We seek to choose L to be a regular language. The regular 
languages are those that can be obtained from the empty language and the singleton languages 
using the operations of union, concatenation, and Kleene star (where K* is the set of all concate- 
nations of or more words from K). Alternatively, regular languages can also be characterised as 
those accepted by deterministic finite state automata. From this latter viewpoint it follows (e.g., 
by the transfer matrix method) that regular languages have rational generating functions (either 
when enumerated by length, or with a separate variable x a for each a € £). We refer readers to 1TT1 
Section 1.4 and Appendix A. 7] for more information on regular languages. 

The following theorem from [ 2 1 demonstrates the connection between subclasses of geometric 
grid classes and regular languages. Essentially, it says that all such classes are extremely well 
behaved. 

Theorem 3.1. Suppose that C C Geom(M) is a permutation class and M is a partial multiplication matrix 
with cell alphabet S. Then the following hold: 

(i) C is partially well-ordered. 

(ii) C is finitely based. 

(Hi) There is a regular language LCE* such that the mapping tp : L — > C is a bijection. 

(iv) There is a regular language L$, contained in the regular language Lfrom (Hi), such that the mapping 
tp is a bijection between L$ and the simple permutations in C. 

Note that the proof of Theorem l3.1l is nonconstructive, so while we use the encoding tp through- 
out this work, we construct the regular languages we use from first principles. 

Albert, Atkinson, and Brignall [3. 4J demonstrate four concrete examples of using these tech- 
niques to enumerate 2x4 classes. 

The examples considered in this paper are inflations of geometric grid classes. The theoreti- 
cal issues of such classes were studied by Albert, Ruskuc, and Vatter [6], who proved that every 
subclass of (Geom(M)) has an algebraic generating function (essentially by showing that it is in bi- 
jection with a context-free language). From this perspective, Section |2] considers the case of a class 
contained in (Geom( l l )), while the next two sections consider classes whose simple permuta- 
tions are contained in more complicated geometric grid classes. 



< di < ■ ■ ■ < d n < 1. 



4. Example #2: Avoiding 4312 and 3142 



We begin our next example with a characterisation of its simple permutations. 
Proposition 4.1. The simple permutations of Av (4312, 3142) and Geom ( ^ 
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Figure 4: The illustration that a simple permutation of Av(4312, 3142) can have no 132 pattern 
after its maximum. Dark grey regions cannot be occupied due to the avoidance conditions, 
lighter regions because of choices made when the entries were selected (topmost, leftmost, etc.). 



Proof. First observe that 

Grid( J I _} ) =Gcom( J J _} ) C Av(4312, 3142), 

so it suffices to prove that the simple permutations in Av(4312, 3142) are contained this grid class. 
Specifically, we will show that in any simple permutation of Av(4312, 3142) the entries that follow 
the maximum make up a "wedge permutation" oriented as < (which is equivalent to avoiding both 
132 and 312), and those preceding the maximum form an increasing sequence. 

So, let a simple permutation tt G Av(4312, 3142) be given. Because tt avoids 4312 there can be no 
312 pattern after its maximum so, for the sake of contradiction, assume that there is a 132 pattern. 
Specifically, choose such a pattern acb where a is as low as possible, and c is as high as possible (for 
the chosen a). This yields the situation depicted on the left in Figure |U Now, in order that the cell 
bounded by {b, c] not form an interval, there must be some entry d in the cell immediately to its 
left. Taking the leftmost such entry yields the diagram on the right in Figure |U In this diagram we 
see that the entries of tt lying in the box bounded by {b, c, d} (including those three entries) form a 
proper interval, contradicting the simplicity of tt. 

We can now argue in a similar fashion that the entries preceding the maximum entry of tt form 
an increasing sequence, i.e., that there cannot be a 21 pattern before the maximum entry of tt. 
Suppose to the contrary that there were one, and choose such a pattern ba where b is as high as 
possible, and a is as low as possible (for the chosen b). Now the cell defined by {a, b} must be 
split either to the left or to the right. The picture on the left of Figure [5] shows that {a, b} cannot 
be split solely to the right, as then taking c to be the rightmost such separator we see that {a, b, c} 
would lie in a proper interval. Similarly, {a, b} cannot be split solely to the left. Thus {a, b} must be 
split on both the left and the right. Now, taking c to be the rightmost separator and d the leftmost 
separator, we have two cases, depicted in the centre and right of Figure [5] In both cases it is clear 
that {a, b, c, d} is contained in a proper interval, and this contradiction completes the proof. □ 

We now consider the encoding cp over the cell alphabet £ = {a, b, C, d} as indicated in Figure|H 
which also shows an example of (p. This mapping is not injective on S* for the following two 
reasons. 

(Gl) The same gridded permutation may be the image of multiple words (in our example, this 
occurs because the pairs {a, b}, {a, d}, and {b, c} "commute", i.e., they may be interchanged 
without affecting the gridded permutation obtained). A method to handle this issue in gen- 
eral (by appealing to the theory of "trace monoids") is presented in Section 7]. 
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Figure 5: The illustration that a simple permutation of Av(4312, 3142) can have no 21 pattern 
before its maximum. In these pictures, the hatched regions cannot be occupied both by the 
avoidance conditions and the choices made when the entries were selected. 




Figure 6: The word acadcdb is mapped by tp to the simple permutation 2473516. 

(G2) A given permutation may have several different ill-griddings. A (nonconstructive) method 
to handle this issue in general is presented in |2j Section 8]. 

In the class we are considering, Av(4312, 3142), it is possible to deal with the issues concretely. 

First we address (Gl). For any particular gridded permutation, we prefer the lexicographi- 
cally minimal word encoding it. For example, suppose that a word contained a factor of the form 
{b, d} + a (here the + superscript signifies that this portion of the word contains at least one letter). 
We could then replace this factor by a factor of the form a{b, d} + and obtain a lexicographically 
lesser word which is mapped to the same permutation. Therefore we forbid factors of the form 
{b, d} + a. The other factor we need to forbid is ca*b (which could be replaced by a factor of the 
form bca*). 

Now we address (G2), which requires us to choose a preferred (geometric) A/-gridding for every 
permutation in Geom(M). Among all Ai-griddings of a permutation, we prefer the one that has 
the most entries in the first column, then the most entries in the second column, and then the most 
entries in the first row. Thus in terms of column divisions 1 = c\ < C2 < C3 < C4 = n + 1 and row 
divisions 1 = r% < r% < = n + 1, we seek to maximise c^, then C3, and then r-2- The words which 
correspond to such griddings can now be characterised as those which do not begin with a*d, b, 
{a, c}*b, or d and are not of the form c{a, C, d}*. 

With this language we may enumerate the grid class itself but we are interested instead in the 
simple permutations. The additional rules for the words encoding simple permutations of length 
at least four are: 



2 This grid class (which, because it can be viewed as a "juxtaposition" in the sense of Atkinson 1 7 |, can be shown to have 
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Figure 7: The shaded areas represent possible intervals in elements of the geometric grid class. 

• To prevent intervals solely contained within an individual cell, we prohibit repetitions aa, bb, 
CC, or dd as factors. 

• To prevent intervals of the form shown in the first pane of Figure [7J we forbid words begin- 
ning with {c, d} 2 . 

• To prevent intervals of the form shown in the second pane of Figure[7l we forbid words of the 

form {a, b, c, d}*{a, c, d}+. 

With these restrictions, we can then use the automata package [10 1 for GAP [12J to count the simple 
permutations of this grid clas^|. For future reference, we record that the multivariate generating 
function for these words of length at least four which begin with a is 

s(x a , x<o, x c , £dj = ~; ! 

1 — X a X c — XbX<j — X c X(j — X a X c Xd — XbX c X(j 

while the words of length at least four which begin with C have multivariate generating function 
x c s(x a , Xb, x c , Xd). Note that our rules preclude words encoding simple permutations from begin- 
ning with b or d. 

Now we characterise the inflations. Because 3142 is simple, it will not occur when inflating a 
3142-avoiding permutation by 3142-avoiding intervals, so we need only avoid 4312. Since the class 
is sum closed, we have that = f 2 / (1 + /), as in Section[2] The skew decomposable permutations 
are a bit more complicated, but divide into a union: 

(Av(21) e Av(312)) U (Av (4312, 3142) Av(12)) , 

where Av0(4312, 3142) denotes the set of skew indecomposable permutations in this class. As the 
intersection of these two is simply Av(21) Av(12), / e = mc + (/ — fe)m — m 2 , where 

x 



basis {2143, 3142, 4132, 4312}) has the generating function 

1 - 6x + tlx 2 - 5x 3 
(1 - x)(l - 3x)(l - 3x + x 2 )' 

3 These simple permutations have the generating function 

x + x 2 — ix z — 3x a 
(l+x)(l-2x) ' 

showing that for n > 3 the simple permutations in this class are counting by the facobsthal numbers (A001045 in the 
OEIS|28]). 
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denotes the generating function for the nonempty decreasing (or, increasing) permutations. Solving 
this shows 

_ m(f + c—m) 

Je — 7~, ■ 

1 + m 

Inflations of simple permutation of length at least four are a bit more complicated, as there are 
several cases. In all such inflations, each entry which corresponds to a b may only be inflated by 
an increasing permutation (but may be inflated by any such permutation), while each entry which 
corresponds to a d may only be inflated by a 312-avoiding permutation (but may be inflated by any 
such permutation). If the word begins with a C, then it follows that the entry corresponding to the 
first C may be inflated by any permutation in Av(312), while each subsequent entry corresponding 
to a C may only be inflated by a decreasing permutation. Otherwise, it follows from our rules 
that the word must begin with an a, and there are two cases. If the entry corresponding to the 
first a is inflated by a permutation containing a descent, then each entry corresponding to a C 
must be inflated with a decreasing permutation. Otherwise, if the entry corresponding to the first 
a is inflated by an increasing permutation, then the entry corresponding to the first C must be 
inflated by a permutation from Av(312), while each subsequent entry corresponding to a C must be 
inflated by a decreasing permutation. From our multivariate generating function for these simple 
permutations, it follows that the contribution of their inflations is 

f-m c c\ cm 2 (c-m + f + cf) 

7 H s(f,m,m,c) = 



f f m J ' ' ' 1 — 2cm — cm 2 — mf — cmf 

Combining this with the generating functions for /® and / e and solving for / yields the generating 
function for the class (or, rather, its minimal polynomial). 

Theorem 4.2. The generating function / for Av(4312, 3142) satisfies 

(x 3 - 2x 2 + x)f 4 + (4x 3 -9x 2 + 6x~ l)f 3 

+ (6x 3 - 12.x 2 + 7x - l)f 2 

+ (Ax 3 - 5x 2 + x)f 

+ x 3 =0. 

The first several terms of this sequence are 

1, 2, 6, 22, 88, 367, 1568, 6810, 29943, 132958, 595227, 2683373, 12170778, 55499358, 

sequence A165538 in the OEIS [20]. Though the form of the equation for / is complicated, the close 
link with the Catalan numbers which can be seen in the previous development is enough to ensure 

that the radius of convergence is quite simple, exactly 1 /5, and in particular fU n — > 5. 



5. Example #3: Avoiding 4231 and 3124 



Before our final example we review a well-studied class. A permutation is layered if it is the direct 
sum of decreasing permutations (these decreasing permutations are called the layers). The class of 
layered permutations has the basis {312, 231}. To restrict the number of layers, we merely need to 
add an additional restriction, of the form 12 • • • k. 



Proposition 5.1. The simple permutations o/Av(4231, 3124) and Geom 




coincide. 
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Figure 8: The end games in the proof of Claim[5j]a. 



Proof. First, it is straightforward to observe that 

Geom f J J ~J ) = Grid f J J ~J ) C Av(4231, 3124), 

so it suffices to prove that the simple permutations of Av(4231, 3124) are contained in this grid 
class. 

Consider a simple permutation tt e Av(4231, 3124) of length n. We analyse the entries of tt to 
the left of n and to the right of n separately, beginning with the entries on the right. 

Claim l5?ri a. The entries to the right of n form a layered permutation with at most two layers. 

Proof of Claim \5.1\ a. By the 4231-avoidance of tt the entries to the right of n avoid 231, so to show 
that they are layered it suffices to show that they also avoid 312. Suppose otherwise. Among all 
occurrences of 312 choose one, cab, in which the 'V and '2' are as close together in position as 
possible (they will in fact be adjacent). Since tt is simple, {a, b} cannot be an interval and thus 
(because they are adjacent) must be separated vertically. We claim that there is at least one such 
separator to the left of n. Let x denote an arbitrary separator of {a, b}. We see that x cannot lie 
horizontally between n and c by 4231-avoidance. If x were to lie horizontally between c and a, 
then {x, a} cannot be separated horizontally anywhere, nor vertically to the right of n, owing to 
the avoidance conditions, so must be separated vertically to the left of n. This separator therefore 
separates {a, b} vertically to the left of n, as desired. The only other case is if x lies to the right of b. 
In this case choose x to be the bottommost such separator. Then {b, x} must be separated. This can 
only occur to the left of n (giving the separator we desire) or to the right of x. In this latter case it 
is easy to see that the region which consists of those points from b to the right and which are also 
vertically between x and b (which contains b, x, and this new separator) can only be separated to 
the left of n, again giving the separator we desired. Therefore in all cases we may assume that there 
is an entry, x, to the left of n which vertically separates a and b. This situation is depicted on the left 
of Figure |U 

Now consider {n, c}. From Figure we see that these entries can only possibly be separated 
vertically by an entry to the left of n and to the right of x. Choose the leftmost such separator and 
label it v. We now have the situation depicted in the centre of Figure [8] However, it is now clear 
that the entries {c, n, v} lie in a proper interval, contradicting the simplicity of tt. This contradiction 
shows that the entries to the right of n must form a layered permutation. 

Having established that these entries are layered, it is easy to see that there are at most two 
layers. Otherwise the entries to the right of n would contain a copy of 123. The '2' and '3' in this 
copy of 123 must be separated by an entry to the left of n (because the entries to the right of n form a 
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Figure 10: Final considerations in the proof of Proposition l5.ll 



layered permutation), but this would create a copy of 3124. This contradiction completes the proof 
of Claim [5TTl a. o 

Claim l5?ri b. The entries to the left ofn and above 7r(n) are increasing. 

Proof of Claim 15. l] b. Let a = iv(n) and suppose to the contrary that the entries to the left of n and 
above a contain an inversion. Choose such an inversion yx with y as far left as possible and x as 
close to y as possible. This gives the situation depicted on the left of Figure As can be seen in 
this diagram, {x, y} could only possibly be separated horizontally. Let z denote a topmost such 
separator. This gives the situation depicted on the right of Figure [9] However, as this diagram 
indicates, y and z now belong to a proper interval, contradicting the simplicity of 7r. o 



Claim l5?fl c. Let c denote the leftmost entry of it greater than n(n) (note that c may equal n). The entries to 
the left of c (which lie below n(n) by Claim [5l\ b) are increasing. 

Proof of Claim \5.1\ c. The proof follows well-travelled lines. Suppose to the contrary that there is an 
inversion among these entries, and choose one such ba where b is as far left as possible, and a is as 
small as possible. The cell bounded by {a, b} can only be split above, and we may choose a split 
point d which is as large as possible. Now the box bounded by {a, b, d} defines a proper interval, a 
contradiction. o 

We are now in position to complete the proof of the proposition, after a brief recap of the struc- 
ture we have established. Let b = 7r(n), let c denote the leftmost entry which is greater than b (note 
that c = n is a possibility), and finally let a denote the bottommost entry which lies horizontally 
between c and n (such an entry need not exist, but this does not affect the argument). We then have 
the situation depicted on the left of Figure [lOl 

The first three labeled regions are further restricted as follows: 
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• Ri must be increasing by Claim l5A] c. 

• i?2 must be increasing by Claim |5A] b. 

• R3 must be decreasing because it avoids 3124. 

Next we claim that R4 and R$ are both empty. First suppose to the contrary that R4 is nonempty, 
and take x to be the topmost entry in this region. We then have the situation depicted in the centre 
of Figure HOl which shows that {b, x} can only be separated by an entry in R5. Let y denote the 
bottommost such separator. It can then be seen that there is no way to separate {b, x, y}. Showing 
that i?5 is empty is very similar. Suppose to the contrary that this region is nonempty and let x 
denote the bottommost entry in the region. It can be seen that there are two ways to separate {b, x}\ 
vertically with an entry in i?5 or horizontally with an entry in R4 . In each case, though, these new 
entries cannot be separated from either b or x. 

Finally, regions Rq and R7 must both be decreasing because 7r avoids 4231 and 3124 respectively. 
Then Claim |5A] a shows that regions i?s and R§ must together form a layered permutation with at 
most two layers. The structure of n is now displayed on the right of Figure [TOl which shows that ir 
does indeed lie in the grid class desired, completing the proof. □ 

Having restricted our attention to this (geometric) grid class, we now seek to place it in bijection 
with a regular language, following points (Gl) and (G2) of Section[4] The first is the easiest to deal 
with since we need only forbid factors of the forms {c, d} + a and d + b. 

To handle (G2), we use the same preference for Af-griddings as in Section |U among all M- 
griddings of a permutation, we prefer the one that has the most entries in the first column, then 
the most entries in the second column, and then the most entries in the first row. The words that 
correspond to these preferred griddings are those which do not begin with b or a*C, do not end 
with d, and are not of the forms d{a, b}* or a*{c, d}+. 

This language allows us to enumerate the grid class itselfl and now we restrict to encoding the 

4 The generating function for this grid class is 

1 - 5x + 7x 2 - x 3 
(1 - x)(l - 2x)(l - 3x) ' 

(sequence A083323 in the OEIS | 20|). Our computations suggest that the basis of this class is 

{4312, 4231, 4123, 3124, 32541, 21534, 21435}, 



but we have not verified this with a formal proof. 
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Figure 12: The shaded areas represent possible intervals in elements of the geometric grid class, 
simple permutations. 

• To prevent intervals contained within individual cells, we prohibit repetitions aa, bb, CC, dd 
as factors. 

• To prevent intervals of the form shown in the first pane of Figure [12j we insist that the last a 
is followed by a b, i.e., we prohibit words which end in a{c, d}*, 

• To prevent intervals of the form shown in the second pane of Figure [12j we require that there 
is a b after the last letter in {c, d}; by the previous two rules, this means we need only prohibit 
words which end in cd or dc. 

• To prevent intervals of the form shown in the third pane of Figure [12J we prohibit words 
which begin with two or more letters from {a, b, c}. 

Finally, we exclude the word deb, which is mapped by <p to the nonsimple permutation 312. 

We can then compute (again using the automata package [10J for GAP [12]) that the multivari- 
ate generating function (counting occurrences of each letter) for the simple permutations in this 
geometric grid class is 

s(x a , Xb, x c , Xd ) = . 

1 x&x\y x^Xq XcX<j — x^XfoXc — x^XcX^ 

It remains only to determine the allowed inflations. First, the sum decomposable permutations all 
have unique representations in the form Av($(312) © Av(4231, 3124), where Av$(312) denotes the 
sum indecomposable permutations in Av(312). As Av0(312) is well-known to be enumerated by 
the shifted Catalan numbers, we get 

f® = (xc + x)f. 

The skew decomposable permutations are of the form Av(312) Av(231, 3124), and so (de- 
viating slightly from our usual conventions) they all have a unique representation of the form 
Av(312) G Av0(231, 3124). The skew indecomposable permutations in Av(231, 3124) are counted 
by the Fibonacci numbers of odd index (1, 1, 3, 8, 21, . . . ), giving that 

_ x - 2x 2 + x 2 
e 1 — 3x + x 2 

We now come to inflations of simple permutations of length at least four. Each entry correspond- 
ing to an a or C may be inflated by an arbitrary member of Av(312). Each entry corresponding 
to a b or non-initial d may be inflated only by a decreasing interval. Finally, the first entry corre- 
sponding to a d (and we see from the multivariate generating function s that every word encoding 
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simple permutations in this class contains at least one d) may be inflated by an arbitrary mem- 
ber of Av(231, 3124), a class counted by the Fibonacci numbers of even index (1, 2, 5, 13, 34, . . . ). 
Therefore, the generating function we are interested in is given by 

x — 2x 2 + x 2 s(c,m,c,m) x — x 2 

f = (XC + X)f + ; -C + 



1 — 3x + x 2 m 1 — 3.x + x 2 

As this equation is linear in /, it is trivial to obtain our final result. 
Theorem 5.2. The generating function for Av(4231, 3124) is 

l- 8x + 20x 2 - 20x 3 + 10x 4 - 2x 5 - (1 - 4x + 2x 2 )yfl - Ax 
2(1 - 3x + x 2 ){-\ + 5x- Ax 2 + x 3 ) ' 

The first several terms of this sequence are 

1,2,6, 22, 88, 363, 1508, 6255, 25842, 106327, 435965, 1782733, 7275351, 29648647, 

sequence A165535 in the OEIS [20J. The radius of convergence is the smallest positive root of the 
cubic factor in the denominator, approximately 0.2451, and hence fn — > 4.0796 . . . 



6. Conclusion 



It should be noted that the three examples presented in this paper are not the only 2x4 classes 
which have been enumerated using these techniques. In [5], the present authors used a precursor 
of this approach to enumerate Av(4231, 1324); in the language of this paper, they proved that the 
simple permutations in this class are contained in 

/ 1 -1 
Geom o • 
\ -1 1 

(Here the • entry denotes a cell in the standard figure filled with a unique point; this notion is 
formally defined in [2, Section 10].) 

Finally, we point out that there may be other 2x4 classes to which these techniques apply. While 
there is a decision procedure to determine whether a given class lies in a monotone grid class (see 
Huczynska and Vatter [13]), there is no known procedure to determine whether a given class lies 
in a geometric grid class (and indeed, there are indications that this question may be quite difficult). 
Needless to say, deciding whether the simple permutations of a given class lie in a geometric grid 
class is expected to be more difficult still. 
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